Processing in Hyperspectral Images (HSIs)

Anna Androvitsanea

anna.androvitsanea@gmail.com

Table of contents

Introduction

Import libraries

Import data for regression

Plots

Spectral signatures for 9 endmembers

Spectral signatures for each endmember

Ground truth

Ground truth masked

RGB Visualization of the 10th band

Part 1: Spectral unmixing (SU)

Scope

In this part I consider the set consisting of the 9 endmembers, where each endmember has a specific spectral signature, which corresponds to the pure pixels in the HSI dataset.

For a given pixel in the image, the aim is to determine the percentage (abundance) that each pure material contributes in its formation and unmix the pixels.

I fit linear regression models that connect the 9 endmembers with the mixed pixels of the HSI dataset. I consider the following models:

(a) Least squares-Least-squares)

(b) Least squares imposing the sum-to-one constraint for $\theta$s,-Least-squares-imposing-the-sum-to-one-constraint)

(c) Least squares imposing the sum-to-one constraint for $\theta$s,-Least-squares-imposing-the-non-negativity-constraint)

(d) Least squares imposing both the non-negativity and the sum-to-one constraint for $\theta$s,-Least-squares-imposing-both-the-non-negativity-and-the-sum-to-one-constraint)

(e) LASSO, imposing sparsity on $\theta$s via $l_1$ norm minimization.-LASSO-impose-sparsity-via-l_1-norm-minimization)

First, I calculate the abundance maps for each material, i.e. 9 maps.

Then I compute the reconstruction error as follows:

I calculate the reconstruction error (for each non-zero class label) of each pixel using the formula: $$error = \frac{||\mathbf{y}_i - \mathbf{X}\mathbb{\theta}_i||^2}{||\mathbf{y}_i||^2}$$ Then, for N pixels I compute the average value: $$\text{reconstruction error} = \frac{error}{N} $$

Finaly, I compare the results obtained from the above five methods based on the abundance maps and the reconstruction error.

(a) Least squares

Plot: $\theta$ vs masked ground truth

Reconstruction error

(b) Least squares imposing the sum-to-one constraint

I am going to apply the cvxopt function in order to introduce constraints to the system of linear regression.

Quadratic systems can be solved via the solvers.qp() function. As an example, I can solve the quadratic problem

\begin{array}{ll} \mbox{minimize} & (1/2) \theta^TP\theta + q^T x \\ \mbox{subject to} & G * \theta \preceq h \\ & A\theta = b \end{array}

In this case I want to minimize the problem:

\begin{array}{ll} ||\mathbf{y} - \mathbf{x}\mathbf{\theta} ||^2 \Rightarrow \\ (\mathbf{y} - \mathbf{x}\mathbf{\theta})^T(\mathbf{y} - \mathbf{x}\mathbf{\theta}) \Rightarrow \\ (\mathbf{y}^T - \mathbf{x}^T\mathbf{\theta}^T) (\mathbf{y} - \mathbf{x}\mathbf{\theta}) \Rightarrow \\ \mathbf{y}^T\mathbf{y} - \mathbf{x}^T\mathbf{\theta}^T\mathbf{y} - \mathbf{y}^Τ\mathbf{x}\mathbf{\theta} + \mathbf{x}^Τ\mathbf{\theta}^Τ \mathbf{x}\mathbf{\theta}\\ \mbox{I ignore the element } \mathbf{y}^T\mathbf{y} \mbox{ since is not subjct to $\theta$}: \boxed{\mathbf{\theta}^T\mathbf{x}^T\mathbf{x}\mathbf{\theta}-2\mathbf{y}^T \mathbf{x} \mathbf{\theta} } \Rightarrow \\ P = 2\mathbf{x}^T\mathbf{x} \mbox{ and } q^T = -2\mathbf{y}^T\mathbf{x} \end{array}

Plot: $\theta$ vs masked ground truth

Reconstruction error ls sum-one

(c) Least squares imposing the non-negativity constraint

Quadratic programs can be solved via the solvers.qp() function as above-Least-squares-imposing-the-sum-to-one-constraint) .

Plot: $\theta$ vs masked ground truth

Reconstruction error ls non-neg

(d) Least squares imposing both the non-negativity and the sum-to-one constraint

Quadratic programs can be solved via the solvers.qp() function as above-Least-squares-imposing-the-sum-to-one-constraint) .

Plot: $\theta$ vs masked ground truth

Reconstruction error ls all-const

(e) LASSO, impose sparsity via l_1 norm minimization

I want to minimize the quantity $(1 / (2 * n_{samples})) * ||Y - X_W||^2_{Fro} + \alpha * ||W||_211$ by imposing sparcity on $\theta$ with L1/L2 mixed-norm as regularizer.

$||W||_{21} = \sum_i \sqrt{\sum_j w_{ij}^2}$ is the sum of norm of each row.

I will implement the class sklearn.linear_model.MultiTaskLasso() which calculates the Lasso linear model with iterative fitting along a regularization path.

Plot: $\theta$ vs masked ground truth

Reconstruction error OLS LASSO

Comparison of regressors

Reconstruction error

Among the regressors I examine, the minimum least squares error is achieved by the ols regressor and the maximum error by ols with the sum-to-one and non-negative constraint for the $\theta_i$ values.

However, I want to see how the regressors perform spatially, i.e. how they reconstruct the map of the campus.

For this scope, I will test each material separately.

Plots

Water

Comparing the ground truth for water against the abundance maps of the regressors I notice the following:

Trees

Comparing the ground truth for trees against the abundance maps of the regressors I notice the following:

Asphalt

Comparing the ground truth for asphalt against the abundance maps of the regressors I notice the following:

None of the five regression models produce a satisfactory result when identifying the roads (asphalt).

Bricks

Comparing the ground truth for bricks against the abundance maps of the regressors I notice the following:

None of the five regression models produce a satisfactory result when identifying the bricks.

Bitumen

Comparing the ground truth for bitumen against the abundance maps of the regressors I notice the following:

Tiles

Comparing the ground truth for tiles against the abundance maps of the regressors I notice the following:

Shadows

Comparing the ground truth for shadows against the abundance maps of the regressors I notice the following:

Meadows

Comparing the ground truth for meadows against the abundance maps of the regressors I notice the following:

Bare Soil

Comparing the ground truth for bare soil against the abundance maps of the regressors I notice the following:

Conclusions

Summing up I get that the following results by comparing the abundance map of each material against the ground truth:

Material best fit second best fit
Water non-negativity constraint sum-to-one - non-negativity-constraint
Trees non-negativity constraint sum-to-one - non-negativity-constraint
Asphalt non-negativity constraint sum-to-one - non-negativity-constraint
Bricks non-negativity constraint sum-to-one - non-negativity-constraint
Bitumen non-negativity constraint simple ols
Tiles sum-to-one constraint sum-to-one - non-negativity-constraint
Shadows non-negativity constraint sum-to-one constraint
Meadows non-negativity constraint sum-to-one - non-negativity-constraint
Bare Soil non-negativity constraint sum-to-one - non-negativity-constraint

Additionally, in the reconstruction error section above, I calculate the following values:

Method Error
Least squares error 0.0015
Least squares sum-to-one error 0.002
Least squares non-negative error 0.0042
Least squares sum-to-one non-negative error 0.0126
Least squares LASSO error 0.0101

The Least squares non-negativity constraint regression error is in the middle of the variance of the errors.

Since it performs the best compared to the other regressors, it comes out on top in this case study.

Part 2: Classification

Import data for classification

Prepare data for classification

Plots

Scope

The task is to assign each one of them to the most appropriate class among the 9 known endmembers (classes).

The classification is perfomed with four pre-chosen classifiers:

The first step-Classification-for-each-classifier) is the performance of a 10-fold cross validation for each classifier. For this step the estimated validation error gets reported.

The second step is the training of each classifier and the evaluation of its performance. This includes the computation of the confusion matrix and the success rate of the classifier.

Finally-Comparison-of-classifiers) I compare the results of the four classifiers.

(A) Classification for each classifier

Naive Bayes classifier

(i) Cross validation

(ii) Confusion Matrix & success rate

Minimum Euclidean distance classifier

(i) Cross validation

(ii) Confusion Matrix & success rate

k-nearest neighbor classifier

(i) Cross validation

(ii) Confusion Matrix & success rate

Bayesian classifier

(i) Cross validation

(ii) Confusion Matrix & success rate

(B) Comparison of classifiers

Estimated validation error

The Bayesian method has the smallest mean value of validation error as well as the smallest deviation.

After that comes the K-nearest neighbor classifier.

Confusion matrices and success rates

The Naive Bayes classifier performs a little more than the average, with a success rate of 66 %. I notice that types of land cover such as trees, bricks, bitumen, shadows and bare soil get identified very well, while some other types are misidentified, ie asphalt is very often identified as meadows or tiles are trees.

The Minimum Euclidean Distance classifier performs just a bit above 50 %. Most types of land cover are misidentified, with the only exception being the Bare soil.

The k-nearest neighbor classifier performs very well with a success rate close to 90%. Bare soil is in all case correctly identified, while most of the types of land cover are also correctly attributed to the relevant pixel. The only outliner is the type Meadows that is identified as Asphalt in many cases.

The Bayes classifier also performs very well with a success rate close to 90%. As before, bare soil is in all case correctly identified. All other types of land cover are also correctly attributed to the relevant pixel. The only outliner is again the type Meadows that is identified as Asphalt in many cases.

The Bayes classifier has the most diagonal confusion matrix of all four classifiers and the most zero or very small non-diagonal values.

Then the k-nearest neighbor classifier follows with a bit more non-diagonal non-zero data.

Reviewing the report of all for classifiers in regard to all classes I notice that:

Reviewing the report of all for classifiers in regard to the overall performance I notice that:

Plots with mixed dataset HSI

Synthesis of plots

In this section I test trained models eith the mixed signature HSI. For these 109 mixed spectral signatures I predict the labels.

I plot all abundance maps together and compare the results per material.

Predictions

Naive Bayes classifier

Minimum Euclidean distance classifier

k-nearest neighbor classifier

Bayes classifier

Water

Comparing the ground truth for water against the abundance maps of the classifiers I notice the following:

Trees

Comparing the ground truth for trees against the abundance maps of the classifiers I notice the following:

Asphalt

Comparing the ground truth for asphalt against the abundance maps of the classifiers I notice the following:

Bricks

Comparing the ground truth for bricks against the abundance maps of the classifiers I notice the following:

Bitumen

Comparing the ground truth for bitumen against the abundance maps of the classifiers I notice the following:

Tiles

Comparing the ground truth for tiles against the abundance maps of the classifiers I notice the following:

In all four cases, the result in identifying the tiles is not satisfactory.

Shadows

Comparing the ground truth for shadows against the abundance maps of the classifiers I notice the following:

Meadows

Comparing the ground truth for meadows against the abundance maps of the classifiers I notice the following:

Bare Soil

Comparing the ground truth for bare soil against the abundance maps of the classifiers I notice the following:

None of the four classifiers are satisfactory in identifying the tiles.

Summing up, taking into account the estimated validation error, the correlation matrices and the abundance maps of the four classifier I conclude that the "winner" is the Bayes classifier, followed by the k-nearest neighbor classifier.

Part 3: Combination - Correlation

I plot the abundance maps of the regressors and classifiers, against the ground truth and discuss the results.

Synthesis of plots

Water

Comparing the ground truth for water against the abundance maps of the regressors and classifiers I notice the following:

A combination of these results could lead to an optimal identification of the course and width of the torrent. In this case I could mask the abundance map of the Bayes classifier with the zero values of the linear regression's abundance map, and obtain the optimum result.

Trees

Comparing the ground truth for trees against the abundance maps of the regressors and classifiers I notice the following:

A combination of these results could lead to the best result with regard to getting read of the misfits at the north-west of the campus. In this case I could mask the abundance map of the linear regression with the non-negativity constraint with the zero values of the k-nearest neighbor classifier's abundance map, and take the optimum result.

Asphalt

Comparing the ground truth for roads (asphalt) against the abundance maps of the regressors and classifiers I notice the following:

A combination of these results could lead to the best result with regard to getting read of the misfits at the north-west of the campus. In this case I could mask the abundance map of the linear regression with the non-negativity constraint with the zero values of the k-nearest neighbor classifier's abundance map, and take the optimum result.

Bricks

Comparing the ground truth for bricks against the abundance maps of the regressors and classifiers I notice the following:

All other results have major errors.

Bitumen

Comparing the ground truth for bitumen against the abundance maps of the regressors and classifiers I notice the following:

Tiles

Comparing the ground truth for tiles against the abundance maps of the regressors and classifiers I notice the following:

All other results have major errors.

Shadows

Comparing the ground truth for shadows against the abundance maps of the regressors and classifiers I notice the following:

Meadows

Comparing the ground truth for meadows against the abundance maps of the regressors and classifiers I notice the following:

Bare Soil

Comparing the ground truth for the bare soil against the abundance maps of the regressors and classifiers I notice the following: